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Abstract 

Nowadays pedagogical testing technology has become the basic tool for diagnosis and assessment of the level of 
students’ mastery of learning material. Primarily they allow testing the acquired knowledge and skills in their use as 
a technology of the definite types of problems solution. Thus, the level of logical reasoning development plays a 
significant role in the successful mastery of many subjects (mathematical courses in particular). So, the problem of 
objective and measurable criteria for assessing the impact of the level of logical reasoning on the mastery of 
mathematical subjects is of current concern. We have studied the scientific sources that describe the testing 
technologies use for assessment of academic achievement as well as the level of logical reasoning development. We 
have found that the existing methods are based only on the individual work of the teacher with a student and don’t 
suggest any diagnosing technology. The goal of our research is to prove the effectiveness of our method as compared 
to the traditional one. 

Keywords: Matrix Test, Diagnosis of logical reasoning level. Testing technology 

1. Introduction 

Academic achievement in Mathematics is influenced by the students’ way of thinking, problem solving skills, 
attitude to mathematics (English L.D., 1997). It means that mastering Mathematics presents specific requirements to 
the level of the students’ logical reasoning. Mathematics is taught as a deductive subject, that doesn’t have other 
grounding for the facts studied but the demonstrative conclusion drawn from a system of definitions. We should note 
that it refers only to the Mathematics study, not to the research in the field of mathematics, where new knowledge is 
generated not only by means of the logical construction. In its turn, such logical-and-deductive aspect of 
Mathematics study has a great impact on the development of the students’ logical reasoning. That’s why much 
attention is given to the development of mathematical thinking on the whole and logical thinking as its essential 
component. In particular the problem of the connection between mathematical (formal) logic and logical reasoning is 
under consideration. 

At the beginning of our research on the links between the level of logical reasoning and the academic achievement in 
Higher Mathematics we suggested that the students should take two tests. Test 1 included tasks that allowed 
measuring the level of proficiency in formal-and-logical constructions. Test 2 consisted of the tasks on mathematical 
analysis. Both tests had the traditional structure and were prepared according to the general requirements to the 
academic achievement tests (Mehrens W.A. & Lehmann I.J., 1987; NCTM, 2000). Similar tests for the assessment of 
the acquired knowledge in logics and mathematical analysis are used, for instance, in Washington University in 
St.Louis (Math-310). The tests results are presented in Table 1. We have distinguished the 4 levels of the test 
completion: low level - less than 40 %, middle level - from 40 % to 60 %, sufficient level - from 60% to 80%, high 
level - more than 80 %. Such unified grading system is used in the Ural Federal University where the one of the 
experimental part of the research was carried out. 
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Table 1. The results of the academic achievement test on the elements of mathematical logics (Test 1) and the basics 
of mathematical analysis (Test 2) 


Test 

The percentage of students who showed certain level of academic achievement 

number 

low 

middle 

sufficient 

high 

1 

8,4 

21,7 

54,3 

15,6 

2 

18,6 

46,9 

30,2 

4,3 


Thus, during the following oral examination the students showed sufficient level of knowledge of the theoretical 
material (more than 60 % of correct answers) that allowed developing a hypothesis that the low level in the Test 2 is 
determined by the insufficient level of logical reasoning development as a means of mathematical problems solution. 
On the one part, mathematical logic is a model of deductive reasoning; on the other it lays claims to being the only 
instrument that allows making a substantiated conclusion of the validity of a given mathematical statement. 
Projecting these roles of mathematical logic into the didactics of mathematical education H. Freudenthal (1973, p. 
617) calls the first one schematization, the second - formalization. Both aspects are based on the corresponding 
logical constructions. Some of them, according to the opinion of H. Freudenthal (1973, pp. 623-627) the most 
important from the point of view of schematization, are also analyzed. An extended list of these constructions with 
the methodological illustrations on their mastering by students is given in the work of E.M. Rekant (2014). It 
discusses: 

1) implicative reasoning (mapping out the logical chain) 

2) case study 

3) apagogic proof 

4) extension or restriction of condition 

5) transition to negation (including counterexamples) and some other logical constructions. 

Students can master these constructions in the process of theorems proving (Yavich R., Gein A. and Gerkerova A., 
2016, p. 102), while studying theoretical material and solving problems independently. It is clear, that in each of 
these processes the mentioned constructions can be combined in different configurations. It is fair to say that the 
more complicated configurations are met in the solved problems the better these constructions are mastered. The 
highest level of complexity is in the so called Olympiad problems in Mathematics (Keller N., Yavich R. & 
Domoshnitsky A., 2014). 

However the problems’ solution cannot be an objective tool for assessment the level of student’s proficiency in the 
given logical constructions. Frequently, a student cannot solve the problem because he cannot adequately reveal the 
logical structure in the problem statement. In many cases the students should restructure the problem statement for its 
successful solution (H.O. Poliak (1987). The impact of the problem statement on its successful solution was 
repeatedly discussed in the works of Silver E.A., 1994, pp. 20, 26; Nicolaou A.A. & Philippou G.N., 2004, p. 654 
and others. But in these works this impact is estimated with the help of questionnaires and standard academic 
achievement tests. In the latter case this impact is estimated indirectly because other factors play a significant role. 
We are not set to lessen the possibilities of the mentioned instruments but in our opinion a valid tool that makes it 
possible to carry out an objective assessment of students’ proficiency in logical construction in the aspect of 
schematization is in great demand. 

2. Matrix Structure in the Test Technology 

The use of test forms in the educational process is regarded one of the major issues of pedagogical science and 
practice of the XX century (Frederiksen N., Mislevy R.J. & Bejar I.J., 1993). The most important positive 
characteristics of test technologies are their objectivity, standardizing of the procedures, and better measurability of 
the outcomes due to the more flexible scale usage. Nowadays test technologies are actively and effectively used for 
the assessment of the students’ proficiency level in the formalization of the logical constructions. Such tests 
explicitly suggest various combinations of logical constructions and it requires validating the conclusions drawn with 
the help of them. Nowadays academic achievement tests are made as standardized tools (Mehrens W.A. & Lehmann 
I.J., 1987; NCTM, 2000). 

To assess the level of proficiency in schematization the student is suggested not just the logical constructions but the 
concrete mathematical derivation on the basis of these constructions. The student should show not only the 
understanding what logical construction is the basis of the given derivation, but also skills in its usage. That’s why 
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direct traditional test approach usage is not effective enough in this case. 

The following example will illustrate the foregoing. 

2.1 Problems 

Problem 1. 

Let A(x, y), B(x, y) h C(x, y) be the predicates in variables x and y. Determine whether the logical formulas are true 

а) ((Vx, y (A(x, y) -> B(x, y)) v (Vx, y (A(x, y) -> C(x, y))) -4 (Vx, y (A(x, y) -> (B(x, y) v C(x, y))); 

б) (Vx, y (A(x, y) -> (B(x, y) v C(x, y))) -> ((Vx, y (A(x, y) -> B(x, y)) v (Vx, y (A(x, y) -4 C(x, y))). 

Problem 2. 

Let f(x) be the function of real variable x with the range of definition X and the range of values Y. Determine 
whether the logical formulas are true 

a) if f(x) is strictly monotone, it is one-to-one correspondence between X and Y; 

b) if f(x) is one-to-one correspondence between X and Y, it is strictly monotone 

As can be readily observed, the logical formulas in Probleml are the formalization of the statements in Problem 2, 
the predicate A(x, y) is interpreted as a ratio of x < y, the predicate B(x, y) as a ratio of f(x) < f(y), the predicate C(x, 
y) as a ratio of f(x) > f(y). The solution of Problem 1 doesn’t require an interpretation, whereas in order to solve 
Problem 2 the student doesn’t need formal logical proof, though without understanding the logical structure of the 
given statements it is impossible to solve the problem. Our technology based on the test approach allows assessing 
the understanding of the logical structure without emphasis on the formal transformation of the logical formulas. 

The test consists of the series of tasks. Every task contains 5-6 questions, whether various mathematical derivations 
are true. Every derivation has a certain basis of logical constructions. Every test task has the following structure: 

Task. State, what derivations are true 

a) (A or B) -> C, 

b) X^ (Y and Z) 

c) (U and V) —» (S orT) 


The task suggests not the logical scheme but the intentional question. But it is required to understand the logical 
scheme of the given derivation to fulfill the task correctly. 

The series of answers are represented by the vector: if we code the correct answer by 1, the absence of answer - by 0, 
incorrect answer by -1, we get a vector with these numbers as its components. This vector allows establishing with 
reasonable certainty, what logical structure the student failed to use. 

The next task suggests the same number of logical structures but with the other intentional questions. In the result we 
get another vector. Then the third task, etc. These vectors placed one below the other form the matrix. Its columns 
clearly show if various logical structures are mastered. 

Of course a question arises: is it possible that the answer is incorrect just because the student doesn’t know the 
material. But it can be easily diagnosed. Suppose we’ve got the following matrix: 

"1 1 0 1 -1 (T 
11101-1 
0101-10 
100-101 
V 1 1-10 0 0 , 

The first column of this matrix shows that the logical structure is mastered, but we have 0 in the third line. It is clear 
that the student doesn’t know the corresponding notion or doesn’t understand its meaning. We can also say that the 
student has difficulty with the construction in the third column. 
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3. Methods of Matrix Test Preparation and the Results of the Experiment 

We prepared Test 3 on the basis of the tasks used for the assessment of academic achievement but in accordance 
with the matrix structure. The use of the tasks of the same types as in Test 2 makes it possible to compare these tests 
on the effectiveness of the diagnosis of the logical reasoning role in the mathematical problems solution. 

Here is an example of the test task (the correct answer is given in the brackets). The full test contains 8 tasks. 

Task 1. Choose all the correct statements 

a) If the function decreases, it is invertible. (Correct.) 

b) If the function is monotone, it is a bijective mapping of its range of definition in the range space. 
(Incorrect; e.g., y = const.) 

c) Suppose, the function is differentiable at each point of its range space. If its derived function is positive 
at each point, the function is increasing. (Incorrect; e.g., y = -l/x.) 

d) Decreasing function may have points at its range space where its derivative is nonnegative. (Correct; 
e.g., y = -x 3 .) 

e) Periodic function can be monotone. (Correct; e.g., y = const.). 

f) Suppose the function is defined and differentiated on a certain interval. This function is monotone, if its 
derivative is not equal 0 at any point of its interval. (Correct). 

Testing was held in several groups of the 1st year students of Ural Federal University (Russia), Ariel University 
(Israel), South Ukrainian National Pedagogical University named after K. D. Ushinsky (Ukraine) a month later the 
classes started. During this period the amount of the material studied wasn’t very large, that made it possible to 
suppose that the level of logical reasoning development will be diagnosed more objectively. 

The matrix of one of the students is given below. The number of the matrix line corresponds to the task number, the 
columns agree to the logical structure of the question in the task. 


f 1 

-1 

1 

1 1 

0 -1 

1 

-1 

0 

1 - 

1 1 

1 

1 

-1 

1 

1 -1 

1 

1 

1 

0 

1 1 

-1 

-1 

0 

1 

1 -1 

1 

-1 

-1 

1 

1 -1 

1 

-1 

1 

1 

1 1 

, 0 

1 

1 

1 

1 1 


We can see that the student has satisfactorily mastered the logical constructions that are used in the tasks a), d), e). In 
fact, these are the simplest constructions of those used in the test. On the whole this student’s level of logical 
reasoning is very modest. 

The results of the test are presented in Table 2. If the student showed low skill in the logical structure analysis in 
more than 50 % of tasks we classified him as belonging to the group of students with insufficient level of the logical 
reasoning development. The students who didn’t cope with more than 50 % of tasks because they didn’t know the 
learning material were assigned to the second group. The third group included the students for whom it was 
impossible to establish with certainty whether the insufficient level of the logical reasoning development or lack of 
knowledge prevented them from passing the test successfully. 

Table 2. The results of the insufficient level of logical reasoning development diagnosis among the students who 
didn’t cope with Test 3 


groups of students 

percentage 

1 

78,3 

2 

12,1 

3 

9,6 
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After the test we conducted a survey asking the students whether they consider such tests useful and if yes, what the 
usefulness of such tests is. The second question was whether they could tell the difference between our test and the 
traditional academic achievement test. 77.3 % of the students responded positively to the first question. We will 
quote two statements as an example: 

Student 1: As a result of the test I have realized that I should pay attention to the revealing of the logical structure 
of the mathematical problems. 

Student 2: The results of the test showed that I can see the logical basics of the problem. I should work more at the 
theoretical material study. 

We will also give the opinion of the teacher who took part in the discussion of the experiment. 

E.M. Recant: The test made it possible to identify the group of the students who have to work at the development of 
their logical reasoning. We will work out a set of tasks that will help them reach the required level of the logical 
reasoning. 

The second question was positively responded by less than 15 % of students. They couldn’t tell the principal 
difference between our test and the traditional academic achievement test. We consider it a favourable characteristics, 
because that means that the students won’t have to adapt to the new type of tests. 

4. Discussion and Conclusions 

1. The experiment allows making a conclusion that the suggested technology of matrix testing makes it possible to 
diagnose the students’ level of logical reasoning development 

2. The advantage of the matrix test over the traditional one is that it allows assessing logical and mathematical 
components simultaneously. 

3. Matrix test makes it possible to diagnose the reason of the student’s learning difficulties. 

4. Matrix test use makes it possible to pay attention to the necessity of spanning the gap in the development of the 
student’s logical reasoning. 

We can repeat the tests on the other material and observe if there is a tendency to the development of the logical 
reasoning. 

The experiment allows making a conclusion that the suggested technology of matrix testing can help diagnose the 
level of logical reasoning development. 

It is also interesting to analyze such matrices of the students’ group. Then we can see an average merit of logical 
reasoning development of the whole group that will allow the correct choice of the teaching methods from the 
perspective of logical reasoning development. 
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